Modularized/layered solutions or "protocols" are known which permit computer systems to communicate, regardless of connection method or vendor-specific hardware implementation, or to permit different networks to communicate or be "internetworked." Known systems provide for connectivity in and among networks of computerized equipment, and address the problems associated with interconnectivity. Layering in known systems divides the task of interconnection and communication into pieces (layers), wherein each layer solves a piece of the problem or provides a particular function and is interfaced to adjacent layers. Each of the layers is responsible for providing a service to ensure that the communication is properly effected. Examples of some services provided by the various layers are error detection, error recovery, and routing among many communication paths. All the layers in conjunction present the overall communication protocol. It is generally well accepted in the art of internetworking that modularizing in layers with well defined functional interfaces, divides and effectively reduces the complexity of the connectivity problem and leads to a more flexible and extensible solution.
A model for describing the layers in a network has been posited by the International Standards Organization (ISO). The ISO open systems interconnection (OSI) model is a seven-layer model, illustrated in FIG. 1. The OSI model provides a standard for describing a network and facilitating computer communications. The OSI model defines the layers and units of information that pass along a network. As illustrated, data from an application or process running on a first host (HOST A) moves down the model network layers to a Physical layer. The Physical layer defines the physical connection which transmits raw bits across a communication channel to another host (HOST B) and up corresponding layers to a process running thereon. OSI, while defining a model or framework in which standards and protocols can be developed at each layer, allows for a flexible approach for implementation of the model. OSI and other layered computer network communications standards are well known and described in detail in the Handbook of Computer-Communication Standards by William Stallings, which is incorporated herein by reference.
Layered protocols and interfaces therebetween have been defined, which provide specifications for communication between a process or program being executed on one computer's operating system and another process running on another computer. Transmission Control Protocol/Internetwork Protocol (TCP/IP) are two protocols that are part of a protocol suite or family of protocols layered and designed to connect computer systems that use different operating systems and network technologies. TCP/IP, which provides a common set of protocols for invocation on dissimilar interconnected systems, is illustrated and mapped in FIG. 1a to analogous layers of the OSI model. TCP/IP is described in detail in ITERNETWORKING WITH TCP/IP, VOLUME I, by Douglas E. Comer, published by Prentice-Hall Inc., 1995, and/or TCP/IP ILLUSTRATED, VOLUME I, by W. Richard Stevens, published by Addison-Wesley, 1994, which are incorporated herein by reference.
TCP/IP is a four layer protocol suite which facilitates the interconnection of two or more computer systems on the same or different networks. In certain networks, such as the Internet, TCP/IP is a requirement for interoperability. The four layers comprise two independent protocols: TCP which can be used to access applications on other systems within a single network; and IP which permits identification of source and destination addresses for communication between systems on different networks.
As illustrated in FIG. 2, application or process data communicated via TCP/IP is "packetized" as it passes down layers through the protocol suite. The original process data first has an information block called a TCP Header prefatorily appended thereto in a TCP layer, to form a TCP packet. The TCP Header contains information to assure that the data travels from point to point reliably without picking up errors or getting lost. An IP layer repacketizes the TCP packet into an IP packet, by adding an IP Header which contains information needed to get the packet to a destination node. The IP packet is further packetized, such as in ANSI/IEEE 802 local area network protocol, with an additional Logical Link Control (LLC) address header and a control header at an LLC layer, to form an LLC Protocol Data Unit (LLCPDU). The LLCPDU is "framed" for transmission by addition of a Media Access Control Header and Trailer, to form a MAC Frame for communication between two TCP/IP facilities.
A considerable amount of "baggage" in the form of headers and trailer information is added to data which is transmitted between facilities using a layered protocol suite, such as TCP/IP and other layered protocols known in the art. Many additional bits are added at the various layers and must be processed for ultimate transmission across a communication channel at the physical layer. At its destination, the transmitted frame must be unpacketized according to embedded instructions and passed upward through the protocol layers to its receiving application or process. In addition to the substantial increase in the amount of information that must be transmitted as a result of packetization in layered protocols, there is a significant amount of processing overhead associated with packetizing data for network and inter-network transmission. Disadvantageously, substantial computing resources and physical transmission media capacity, representing real costs, must be involved in the generation and application of header and trailer information associated with putting data and all its protocol suite baggage through the communication channel.
Historically, early networks were constituted by a plurality of computers daisy chained in a limited physical space by a common physical medium or wire, e.g. Ethernet. The primary concern with Ethernet resulted from the simultaneous transmissions on the common wire, which resulted in "collisions" and limited the amount of information that could be transmitted over the medium. When collisions occur, all packets involved in the collision are lost and must be re-transmitted. Thus Ethernet interfaces within a Local Area Network (LAN) were designed to include collision avoidance mechanisms. In this manner, traffic on the network was detected in order to await a safe opportunity for transmission. Accordingly, the amount of information that could be successfully transmitted over the LAN in a given amount of time, referred to as "bandwidth," was increased.
As LANs grew, hardware components were required and developed (e.g. repeaters), to convey the data signals intelligibly along the extended path. Repeaters merely passively amplified signals passing from one network cable segment to the next. While repeaters increased the physical distances over which network data could be transmitted, they did not contribute to any increase in network bandwidth.
Hardware "bridges" effectively replaced repeaters for extending the size and scope of networks. Bridges addressed optimization of connectivity and, to an extent, enhanced network bandwidth. In contrast to repeaters, bridges effectively isolated network segments by actually re-creating a packet of signals as it is forwarded in a single network. Bridges are comprised of input and output ports, and maintain tables which map physical addresses to particular ports of the bridge. The tables are based on Data Link Layer (OSI Model level 2) information in each data packet header. The bridge maps an incoming packet for forwarding to a bridge output port based on the packet's destination address. Bridges, like Ethernet interfaces, employ collision avoidance mechanisms at their ports, so they can enhance bandwidth by ensuring that simultaneous transmissions in isolated bridged segments do not collide. Forwarding via bridges, however, introduces substantial delays or latencies in network transmissions as the packets are processed for forwarding. Also, memory requirements for maintenance of tables in bridges is substantial as traffic and number of nodes in a network increases.
Bridges topologically configured on a single level to connect network segments, as illustrated in FIG. 3a, may actually negatively impact bandwidth. Data traversing the network from a source in segment #1 to a destination in segment #4, must pass through intermediate segments #2 and #3. This effectively reduces the bandwidth available to systems residing on segments #2 and #3. A solution to this effective reduction in bandwidth was introduced with the concept of a network "backbone," as illustrated in FIG. 3b.
The physical extension of single networks was soon followed by the need to physically and logically connect distinct networks, i.e. internetworking. Providing optimal connectivity, maximizing the amounts of information conveyed ("bandwidth"), and facilitating interaction or isolation between interconnected networks became (and continue to be) paramount considerations as the size and scope of networks and Internetwork increased.
Internetworking, or interconnection of discrete or logically distinct networks, was facilitated by the introduction of routers. Separate LANs, each having their own logical address spaces, are interconnected by routers to form "subnetworks" or "subnets". Subnet configurations effectively partitioned networks and avoided the problems associated with management of very large address spaces in large networks.
Routers operate on the Network Layer information (OSI Model level 3, IP packet level in TCP/IP) and therefore facilitate transmission of information among and between different subnet protocols. Isolation of subnets via routers localizes collisions and simplifies the implementation of subnet broadcasts. Routers enabled the configuration of complex network topologies while enhancing bandwidth and facilitating interconnectivity. However, known routers, like bridges, require large amounts of memory to maintain routing tables, and disadvantageously introduce latencies in the transmission of information as it is processed at the appropriate protocol stack layer.
Complexities in network configurations led to the implementation of hierarchical network topologies, and created the need for flexibility in reconfiguring existing networks. "Hubs" or "concentrators" (different names for effectively the same type of device), were developed to provide central facilities for network physical interconnections. Hubs essentially receive the wiring/interconnections for all of the systems or nodes configured in a subnet (i.e. one node per hub port), and eliminate the daisy-chaining of connections between systems in a network. Hubs can be centrally located, such as in a network cabinet or telephone closet, such that patching between hubs or subnets can be easily implemented.
Switches have been developed more recently, and are increasingly more popular than hubs. Switches, as opposed to hubs, actually process the network traffic or packets and, like bridges, switches maintain tables which map physical addresses to particular ports of the switch. The switch tables are based on Data Link Layer (OSI Model level 2) information in each data packet header so that incoming packets are forwarded to a switch port based on the packet's destination address. Switches are effectively multiport bridges, typically with enhanced capabilities that permit them to function as routers. Typical switches have fast backplanes for receiving signals from nodes and either use a matrix of connections between every port connection possibility, or a central memory table repository, to effect store and forward operations for network traffic. Switches, like bridges and routers, introduce latency in network communications.
Communications over internetworked computers (generally referred to hereinafter, irrespective of the physical links, as "telecommunications"), in many implementations is based on the concept of switching. In telecommunications generally, switching determines which path a data stream takes as it traverses the network(s) from a source node to a destination node.
Traditional "packet switching" schemes (such as the widely deployed "X.25" technology designed under the auspices of the international standards organization CCITT), require that the source and destination nodes be in communication, i.e. logically connected, before any packets can be transferred. This is referred to as a connection oriented protocol. Each packet requires an identifier for its destination connection. Entire packets have to be sent from point to point. At each receiving point the packet is typically checked and an acknowledge is returned to the sending node before the packet is sent to another point toward its destination connection. If an error was detected in the received packet, the packet would be discarded, a control packet would be returned to the sending node and the packet would have to be re-transmitted. Thus, significant amounts of buffer memory are required in packet switching implementations to buffer the packets in transit. Significant delays are introduced by the fact that full packets have to be buffered and checked before being further transmitted.
Routers and switches, which connect networks using the same Transport Layer protocols but different Network Layer protocols, provide "connectionless" data transfers. In contrast to connection oriented, packet switched implementations, packets in connectionless router/switch implementations contain the address of their destination and therefore do not require a logical connection established between transferring nodes. It should be noted that with the TCP/IP protocol suite the destination node verifies that the packet is complete and correct, and requests re-transmission if necessary. TCP/IP can be used over connectionless or connection-oriented environments.
Routers and switches connected in Wide Area Networks (WAN) contribute to possibly the single most severe network issue--limited backbone scaling--in contemporary Internetwork (such as the Internet). This problem is sometimes referred to as the "N-1 problem." The problem arises from the fact that each independent aggregate entity, i.e. subnet or "domain", controls the allocation of sub-network (IP) addresses. Consequently, once inter-connected to the backbone, fine gradient sub-network level detail populates the forwarding table of every backbone switch or router (the terms "switch" and "router" are used effectively interchangeably hereinafter). Since the amount of memory consumed by the forwarding tables is a function of the number, N, of nodes in subnets, more and more memory is required for the forwarding tables of the backbone routers as compared to interior domain routers. Thus a practical limit is imposed on the ability to add subnets to a backbone. Beyond that practical limit, the performance of the network is actually better if there is one less node. To interconnect N items with a distinct "end to end" path or connection, .SIGMA.(n-1) or n (n-1)/2, links are required. As an example of how quickly this information grows, 5 nodes require 10 total links, 50 nodes require 1225 total links, and 500 nodes require 124,750 total links.
The issue of sub-network information scaling requires increasing backbone switch memory as feeder switches inter-connect to more network domains or subnets. The inability to increase memory due to practical cost/physical limitations, substantially negatively impacts backbone performance in a number of ways. Since the forwarding table size in routers/switches connected to the backbone is finite, table entries are typically cached in and out of the memory, slowing down performance. For the period that a table entry is not in the forwarding table, the data will be dropped at the switch. Thus if the sub-network is an aggregate of higher scale, i.e. a subnet with a large quantity of nodes or other subnets, larger volumes of discarded packets occur, when not in cache. Exacerbating the problem is the ever increasing demand for bandwidth, which can only be met with highly scaleable networking technology.
Furthermore, known routers/switches employ traffic optimization algorithms that are typically based on the concept of directing traffic to the shortest path first. Disadvantageously, such "shortest-path-first" router models tend to have the opposite of the desired effect in that they lead to undesirable congestion. When each node picks the shortest path, independent of what other routers are doing, the network will tend to congest nodes/links with the highest connectivity at the center of a network, first. This is primarily due to the fact that the shortest-path-first algorithm is based on a two dimensional model. Accordingly, the most connected nodes will have the shortest paths to the most nodes, which will make them the most likely nodes to be selected by each independent node space implementing its respective shortest-path-first optimization algorithm. Since each node selects a path, independent of other nodes and what they are selecting, a link will congest before that congestion is recognized and the routers determine another route. In addition, since each router typically has a mechanism to exchange feedback about a congested node, each router will spin-off to calculate another path to choose, all using the same algorithm. This will tend to merely move the congestion to another node. This also tends to cause routers to calculate routes more often, while the congestion moves in an orderly fashion around, or back and forth with each new route calculation.
Various solutions have been proposed to address the need for higher levels of scalability in internetworking, with the benefit of eliminating the possibility of congestion at any particular link. Asynchronous Transfer Mode (ATM) technology is relatively highly scaleable and upholds the promise of providing optimal connectivity, maximizing bandwidth, and facilitating interaction or isolation between interconnected networks. ATM, however, is perceived by many as being perhaps one of the most complex networking technologies introduced to date.
ATM is a connection-oriented implementation that is structured around ATM "cells" and cell switches. An ATM network includes ATM switches which are interconnected by point-to-point ATM links/interfaces. Prior to any data transfer a connection must be established between the ATM source and the ATM destination (i.e. ATM is connection-oriented). ATM involves ATM specific signaling protocols and addressing structures, in addition to protocols required to route ATM requests across the ATM network. As is appreciated in the art, these ATM specific protocols affect the manner in which existing higher level protocols can operate over ATM networks. That is, it is generally not a simple matter to implement ATM in the context of existing network configurations.
ATM is a hardware specific implementation comprising ATM switches that support two kinds of interfaces: user-network interfaces (UNI); and network-node interfaces (NNI). UNIs involve one type of ATM cell or information format, and connect ATM end-systems, such as hosts, routers etc., to an ATM switch. NNIs involve a second type of ATM cell and generally connect an ATM switch to an ATM switch. Virtual circuits are set up across an ATM network to effect the connections for making data transfers.
Two types of virtual circuits can be set up in ATM networks: virtual paths which are identified in a cell by virtual path identifiers (VPI); and virtual channels which are identified by virtual channel identifiers (VCI). VPI and VCI are only significant across a particular ATM link, and are remapped as appropriate at each switch. Thus each ATM switch receives a cell across a link according to a VCI or VPI value. Each switch maintains a local translation table in order to look up the outgoing port(s) of the connection and to insert a new VCI/VPI value. The ATM switch then retransmits the cell on the outgoing link with the new connection (VCI/VPI) identifiers.
The hardware specific implementation of ATM presents numerous disadvantages apart from its complexity. As with conventional (non-ATM) router s and switches ATM switches must deconstruct and reconstruct information (cells) as the information traverses the network, which introduces significant latencies. As with other connection-oriented protocols latencies are introduced by the fact that full cells of information must be sent, received and checked before being forwarded toward their ultimate destination. In effect, increases in bandwidth obtained with ATM are gained by increasing switching speeds. The substantial amounts of protocol related baggage transferred with data over ATM links likely will run up against real, hardware imposed limits in switching speed/bandwidth.